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Abstract 


Currently, most of the displays in control rooms can be categorized 
as status screens, alerts/procedures screens (or paper), or control 
screens (where the state of a component is changed by the operator). 
The primary focus of this line of research is to determine which pieces of 
information (status, alerts/procedures, and control) should be collocated. 
Two collocated displays were tested for ease of understanding in an 
automated desktop survey. This usability study was conducted as a 
prelude to a larger human-in-the-loop experiment in order to verify that 
the 2 new collocated displays were easy to learn and usable. The results 
indicate that while the Dial-on-Control display was preferred and 
yielded better performance than the Multi-Dimensional Object display, 
both collocated displays can be easily learned and used. 


Introduction 

Currently, most of the displays in control 
rooms can be categorized as status screens, 
alerts/procedures screens (or paper), or control 
screens (where the state of a component is 
changed by the operator). With the advent and 
use of graphical displays and various types of 
input devices, and the associated computing 
power available to compute and display 
information, it is now possible to combine these 
different elements of information and control 
onto a single display. The primary focus of the 
present research is whether these pieces of 
information (status, alerts/procedures, and 
control) should be collocated. 

Previous research found that operators like to 
have status, alerts and procedures, and controls 
located on the same screen or have status and 
alerts and procedures on one display with 
controls on another (Bartolone & Trujillo, 2002, 
p. 2; Trujillo, 2001a, 2001b). This research was 
done with displays that were not specifically 
designed for collocation. A follow-on 
experiment was planned to test two displays 
specifically designed for collocation. 

Before the follow-on experiment was run, the 
collocated displays were tested for ease of 
understanding by way of an automated desktop 
survey. This pre-test was conducted to ensure 
that the two newly designed collocated displays 
were intuitive enough such that the general 


untrained population could begin using and 
understanding them with just a brief introduction 
to them. 

Objectives 

This experiment was conducted to attest that 
subjects could quickly and easily understand the 
two new collocated displays. In order to fully 
meet the objectives, three independent variables 
were varied. These independent variables were 
(1) display format, (2) parameter repetition, and 
(3) display order. 

For display format, each subject saw the 
baseline display and one of the collocated 
displays. The collocated displays were the dial- 
on-control (DC) and multi-dimensional object 
(MDO) display formats. 

Parameter repetition and display order were 
manipulated in order to satisfy the objective that 
the display formats were quickly and easily 
understood. Parameter repetition referred to the 
number of times each subject saw a particular 
non-normal parameter (pressure, temperature, 
and quantity). Display order referred to the 
order subjects saw the baseline display and the 
collocated display. 

Display Format 

Each subject saw two display formats: (1) 
standard status displays and controls (baseline) 
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and (2) one of the collocated displays - dial-on- 
control or multi-dimensional object. All the 
display formats modeled the same system. 

The system modeled was generic and 
consisted of a tank (TANK) that fed two pumps 
(L PUMP and R PUMP) whose combined 
output was shown with overall system 
parameters (SYS) (Fig. 1). The parameters 
associated with each component and their alert 
levels are shown in table 1. For this study, 
warning alerts were red in color, cautions were 
amber, and advisories were cyan. Normal 
values, which comprised the remaining 
instrument range, were shown in green. 

Baseline 

The baseline display format presented the 
status information separate from the control 
screen. Status information was represented with 
standard dial formats (Fig. 2). When all the 
parameters were at their expected values, the 
dial pointers were horizontal. A decreasing 
value was indicated with the dial pointer rotating 
counter-clockwise while an increasing value was 
indicated with the dial pointer rotating 
clockwise. This aspect of the display 
encouraged check reading because pattern 
matching could be employed; any parameter 
deviation had a dial pointer departing from 
horizontal (Sanders & McCormick, 1987, pp. 
128-129). 

The control screen duplicated the functional 
layout of the generic system (Fig. 2). 
Components that had no change of state, in this 
case the tank and system information, were 
shown with white squares. Components that 
could change state (i.e., turn “on” and “off’), 
such as the left and right pumps, were 
represented with circles. A single outlined circle 
indicated a component was “on” while a double 
circle denoted a component was “off.” The 
outline color of the component announced the 
highest alert range the component’s parameters 
had reached. In other words, if the left pump’s 
pressure was in the warning range and 
temperature was in the caution range, the 


component outline was red for warning. A 
“failed” component was shown with a red 
outline and a red X across the component. 

Collocated Displays 

The collocated displays were designed so 
that all three types of information were located 
on one screen (Bartolone & Trujillo, 2002; 
Mahaffey, Horst, & Munson, 1986, p. 1514; 
Trujillo, 2001a, 2001b) and judiciously used 
color in order to enhance the displays (Stokes, 
Wickens, & Kite, 1990, pp. 65-87). Both were 
pictorial in format, which suggested less 
processing would be required (Weinstein & 
Wickens, 1992, p. 137) especially if patterns 
could be learned and discerned (Stokes, 
Wickens, & Kite, 1990, p. 9). Furthermore, the 
information in a single location could enhance 
emergent features (Buttigieg & Sanderson, 1991, 
p. 647) and increase the likelihood of noticing a 
non-normal situation developing (Davis, 2004, 

p. 2). 

Dial-on-Control (DC). The dial-on-control 
format was a collocated display with the 
parameter information integrated into the control 
display (Fig. 3). This display shared some of the 
conventions employed in the baseline display. 
Components with no change of state were 
depicted as square while components that could 
change state were depicted as circles. Also, a 
single outlined circle indicated a component was 
“on” while a double circle designated a 
component was “off.” 

Each component symbol was split in half 
vertically. The left half of the component 
symbol registered either pressure or quantity 
while the right half of the component symbol 
indicated temperature. Pressure was shown with 
a triangle icon, quantity with a rectangle, and 
temperature with a circle. The icons traveled 
around the component outline. As a parameter 
increased, the icon rose and as a parameter 
decreased, the icon fell. When all the 
parameters were at their expected values, the 
icons were at the horizontal middle of the 
component outline. Therefore, this display 
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incoiporated both collocation and limited pattern 
matching. 

The appropriately color-coded alert range 
was indicated at either the top or bottom of the 
component outline. For example, the tank’s low 
quantity warning was shown on the lower part of 
the left-half of the tank outline and the tank’s 
high temperature advisory was shown on the 
upper part of the right-half of the tank outline. 
The rest of the component outline was green, 
except for the alert ranges. 

If a parameter reached an alert range, the 
icon changed from white to black and the 
component name was in the same color as the 
highest classification that the component’s 
parameters were in; otherwise the component’s 
name was in white. A “failed” component had a 
red X through the component and the 
component’s name was in red, which indicated a 
warning. 

Multi-Dimensional Object (MDO). As 

with the DC display, the MDO display 
collocated the parameter information with the 
control display but the parameter information 
was more integrated pictorially into the control 
display (similar to Albert et al., 2003); therefore, 
this display supported collocation with no 
pattern matching because subjects were 
unfamiliar with this display (Fig. 4). The 
additional incoiporation of the parameter 
information was thought to enhance visual 
processing of the display in a glance such as was 
found with polar-star displays (Bartolone & 
Trujillo, 2002; Mahaffey, Horst, & Munson, 
1986; Trujillo, 2002, 2004). As with the other 
two displays, components with no change of 
state were square while components that could 
change state were circles. For the components 
with a change of state (i.e., turn “on” and “off’), 
a solid white outline indicated a component that 
was “on” while a dotted white outline indicated 
a component that was “off.” A “failed” 
component had a red X through it. 

Pressure was indicated by size. If pressure 
increased, the colored component fill grew 


proportionally (Fig. 5). If pressure decreased, 
the colored fill shrank proportionally. The 
beginning of a pressure alert range was shown 
with a dotted colored outline indicating the alert 
level (red, amber, or cyan). If the pressure alert 
range was reached, the dotted colored outline 
went solid. 

Temperature was indicated by fill color (Fig. 
6). If the temperature increased, the fill color 
changed from green to the alert range color from 
the center out. If the temperature decreased, the 
fill color changed from green to the alert range 
color from the outside in. The beginning of the 
high temperature alert range was indicated by 
the outside edge of the colored component fill 
and the beginning of a low temperature alert 
range was indicated by the center of the colored 
component fill. If a high temperature alert were 
reached, the fill color was the same as the alert 
range color with a dotted green outline at the 
edge. If a low temperature alert range were 
reached, the fill color was the same as the alert 
range color with a small black circle in the 
middle. 

Quantity was indicated by fill level (Fig. 7). 
The fill level would rise if the quantity increased 
and the fill level would fall if the quantity 
decreased. Normal fill level was indicated by a 
small white horizontal line on the side of the 
component outline. The beginning of an alert 
range was shown by a small color coded line on 
the side of the component outline. When an 
alert range was reached, the component name 
turned black and the top of the fill level changed 
to the color coded alert range. 

Hypotheses 

The general hypothesis was that the two new 
collocated displays would not be significantly 
different from the baseline display with respect 
to operator understanding of the displays and 
preference. The specific hypotheses are below. 

When considering the display types, the 
following statements hold true: 
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1. When choosing the component affected, the 
direction of parameter movement or the alert 
level of a parameter, there will be no 
accuracy difference among the three 
displays. 

2. When choosing the parameter affected, there 
will be no accuracy difference between the 
baseline display and the DC display but 
there will be a small decrease in accuracy 
for the MDO display from the baseline 
display. 

3. Subjects will be able to discern the state of 
the component (i.e., “on,” “off’ or “failed,” 
and changeable or static), the component 
affected, the parameter affected, the 
direction of parameter movement, and the 
alert level with the minimal training 
provided. 

4. Subjects will have no preference between 
the baseline display and the DC display but 
they will have a preference for the baseline 
display over the MDO display. 

5. Workload will be no different between the 
baseline display and the two collocated 
displays. 

6. Subjects will indicate the ease of 
determining the component affected, the 
parameter affected, the direction of 
parameter movement, the alert level, and the 
overall ease of use for the two collocated 
displays to be equal to the baseline display. 

When considering the order, the following 

statement holds true: 

7. There will be no significant difference 
between the order the subjects saw the 
displays. 

Experiment Design 

Subjects 

Ten people participated as subjects. Four 


were female and 6 were male, and 2 were 
certificated pilots (1 female and 1 male). All 
used computers for work and were familiar with 
flight decks and flight procedures even if they 
were not certificated pilots. 

Test Design 

The experiment was conducted on a PC using 
an HTML interface. The HTML code was 
hosted on an SGI® IRIX® machine acting as the 
web server, which used Netscape© 
Administration web server software version 
2.13. The HTML code was programmed in 
PERL version 5.6.1. 

Each subject saw the baseline display and 
one of the collocated display formats. Four 
subjects saw the baseline display first and then 
the collocated display (2 baseline then DC and 2 
baseline then MDO) while the rest of the 
subjects saw the collocated display first and then 
the baseline display (3 DC then baseline and 3 
MDO then baseline). 

Component and parameter order was 
randomized by subject. In general, each subject 
had 15 non-normal conditions for the left pump, 
14 non- normal conditions each for the right 
pump and tank, and 7 non-normal conditions for 
the system. Subjects also had 20 non-normal 
conditions involving pressure, 6 non-normal 
conditions involving quantity, and 24 non- 
normal conditions involving temperature. 

Each subject answered 15 questions per 
display type about the symbology and 25 
questions about a non-normal condition they 
saw per dial type. This resulted in the study 
lasting 40- 60 minutes per subject. 

Dependent Variables 

The dependent variables consisted of the 
subjects’ answers to the questions. The 
objective questions asked and the answer 
choices are listed in Table 2. 

At the end of each display format, subjects 
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completed the NASA-TLX workload measure 
questionnaire (Byers, Bittner, & Hill, 1989; Hart 
& Staveland, 1988). Each of the individual 
factors that constitute the NASA-TLX (mental, 
physical, and temporal demand; performance; 
effort; and frustration level) was rated on a scale 
with nine equal graduations. 

Next, subjects indicated how difficult it was to 
determine the non-normal situation (Questions 1 
-5). At the end of the test, subjects answered 
questions comparing the collocated display to 
the baseline display (Questions 6 - 9). Lastly, 
subjects answered questions about their 
preferences for the collocated displays 
(Questions 10 - 11). Table 3 details the 
questions. 

Procedure 

When subjects first arrived, they filled out an 
informed consent form (Appendix A). They 
were then briefed on the hardware and software 
setup. Subjects used a right-handed mouse and 
the web-interface was shown on a 53.34 cm 
(21”) (50.292 cm (19.8”) visible) diagonal Sony 
Trinitron™ flat screen. Display resolution was 
1280 x 1024 pixels with a 32 bit color quality. 

Each display description was self paced. 
Subjects first read a description about the 
display format they would be seeing. They then 
answered questions 1-4 from Table 2 for each 
of the parameters (pressure, temperature, and 
quantity). 

Once the subjects answered the above 
questions correctly, they then read directions 
about the next phase of the task. During this 
phase, a picture was displayed for 3 sec. On the 
picture was a component with a single parameter 
deviating from normal (fig. 8). After 3 sec, the 
display was replaced with questions 5-7 from 
Table 2. Each subject had 25 pictures for each 
display format. 

After the subjects answered the questions for 
timed a display format, they completed the 
NASA-TLX and indicated how difficult it was 


to determine the non-normal situation. 

Next, the display format description for the 
second display seen was presented and the above 
process was repeated. 

Lastly, subjects answered the questions 
comparing the collocated display to the baseline 
display and their overall preferences. 

Data Analysis 

Data was analyzed using SPSS R vl3.0, 
Statistical Product and Service Solutions (SPSS 
Inc., 2004). The data was analyzed using a y} 
test except for parameter repetition which was 
analyzed using a 1-way ANOVA. In all cases, 
statistical significance was set at p<0.05. 

Results and Discussion 

Accuracy in Identifying Affected 
Characteristic 

Component 

Display format was significant (% 2 (2)=9.89, 
p<0.01) for determining which component (tank, 
left pump, right pump, system) had the non- 
normal value. Subjects using the MDO display 
format were 92% accurate in determining the 
component that was not normal as compared to 
the baseline and DC display formats, 99% and 
97% accuracy respectively. 

Parameter 

Display format and display order were 
significant (% 2 (2)=25.02, p<0.01 and 

X 2 (3)=15.70, p<0.01 respectively) for 

determining which component parameter 
(pressure, temperature, quantity) was affected. 
Subjects using the MDO display format were 
85% accurate in determining the component 
parameter that was not normal as compared to 
the baseline and DC display formats, 98% 
accurate and 94% respectively. Although not 
significant, temperature caused the most 
problems, especially for the MDO display 
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(Figure 9). Also, subjects were correct the least 
often when they saw the MDO display first and 
then saw the baseline display. See table 4 for a 
detailed breakdown. 

Direction 

Display format (x 2 ( 2)=43.62, p<0.01), 

display order (x 2 (3)=l 6.70, p<0.01), and 

parameter repetition (F(ll)=1.92, p<0.04) were 
significant for determining which direction the 
non-normal parameter was moving. Subjects 
using the MDO display format were correct in 
determining if the component was high or low 
only 81% as compared to 98% and 96% for the 
baseline and DC display formats. Subjects were 
best at determining the direction when they saw 
the DC display first (Table 5). As for repetition, 
the first two had an accuracy level of 85% while 
the other ten repetitions had an accuracy level 
above 90%. 

Alert Level 

Display format and the order of the display 
formats were significant (x 2 (2)=96.13, p<0.01 
and x 2 (3)=43.83 p<0.01 respectively). For 
display format, subjects were only accurate with 
the alert level 66% of the time for the MDO 
display format as compared to 98% and 97% 
accurate with the baseline and DC display 
formats respectively. Subjects were correct the 
least often when they saw the MDO display 
format first (Table 6). 

Overall Correctness of Answer 

The overall correctness of each subject’s 
answer was rated by the following: 4 for 
answering the correct component, parameter, 
direction, and alert; 3 for answering the correct 
component, parameter, and direction; 2 for 
answering the correct component and parameter; 
1 for answering the correct component; and 0 for 
not answering anything correctly. 

Display and the order seen were significant 
(X 2 (2)=107.72, p<0.01 and x 2 (3)=49.62, p<0.01 
respectively). The MDO display had the lowest 


level of correctness (Figure 10 and Table 7). 
The biggest drop in correctness was seen when 
the MDO display was seen first (Figure 1 1 and 
Table 7). The most accurate recollection of the 
non-normal situation occurred when subjects 
saw the DC display first. 

Training 

Display format and display order were 
significant in determining whether a parameter 
was high, low, or in an alert range during 
training (x 2 (2)=9.72, p<0.01 and x 2 (2)=8.08, 
p<0.05 respectively). For display format, 
subjects primarily had trouble with the MDO 
display for determining an alert (Figure 12 and 
Table 8). One subject had difficulty determining 
a high pressure with the MDO display. For 
determining an alert, subjects had difficulty 
determining a pressure alert for the DC and 
baseline display (1 incorrect answer for each), 
and temperature (3 incorrect answers) and 
quantity ( 1 incorrect answer) alerts for the MDO 
display. For display order, seeing the DC or 
MDO display first had the most errors (Figure 
13 and Table 8). 

Subjective Data 

Workload 

There was no statistically significant 

difference in workload across the three displays. 
Therefore, this suggests that subjects perceived 
that the 2 collocated displays did not add 
additional workload although the MDO display 
did have a slightly higher workload than the 
other two displays (Figure 14). 

Ease 

There was no statistically significant 

difference between their ratings of how easy or 
difficult they thought it was to determine the 
component and parameter affected, whether it 
was high, low or in an alert range, the type of 
alert, and determining the overall state of the 
affected component by display type. A trend for 
determining if the parameter was high, low, or in 
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an alert range was present for the displays (% 2 
(2)=5.76, p<0.06). The MDO display had a 
higher difficulty rating than the baseline and DC 
displays (Figure 15). 

Compare 

There was no statistically significant 
difference for dial preference. A large 
difference between preferences for DC and 
MDO displays when compared to the baseline 
display was seen but was not statistically 
significant due to a small sample size and the 
standard deviation (Figure 16). 

Concluding Remarks 

This usability study was conducted as a 
prelude to a larger human-in-the-loop 
experiment in order to verify that the 2 new 
collocated displays were easy to learn and 
usable. This objective was accomplished by 
conducting a brief training period followed by 
several runs where subjects had to remember a 
single-point failure after looking at a static 
display for five seconds. 

The better performance for the baseline and 
DC displays is not unexpected. The baseline 
display setup is familiar to most people. The 
DC display incoiporated several aspects of the 
baseline display into it; dial-like parameter 
readings were still available, which enhanced the 
pattern matching aspect of the display for 
detecting non-normal situations (Jones, 
Wickens, & Deutsch, 1990, p. 2; Sanders & 
McCormick, 1987, pp. 128-129). Also, the 
amount of perceptual information of the two 
displays were about the same so there may have 
been no detriment associated with attending to 
separated displays as opposed to looking at only 
one object (Davis, 2004, p. 5) 

Furthermore, this task was essentially a 
detect and diagnose task. The dial-type formats 
of the baseline and DC displays may have 
facilitated this task because effective diagnosis 
may require separate displays, in this case 
separate dials. This may be because attention 


must be paid to each cue in order to make the 
proper diagnosis (Jones, Wickens, & Deutsch, 
1990, p. 2). To further aid in the detection of 
these pertinent parameters, other research has 
suggested that less integrated displays may be of 
benefit (Davis, 2004, p. 15). In any case, a 
change from the expected normal slows 
detection. Recognition of the non-normal 
situation requires more processing to be done 
(Hawkins & Blakeslee, 2004, p. 167) because an 
operator’s expectation no longer matches what 
he sees (Hawkins & Blakeslee, 2004, p. 88). 

On the other hand, the two collocated 
displays, DC and MDO, showed no statistically 
significant differences to the baseline display 
subjectively. The perceived workload was the 
same for all displays. The MDO display’s 
performance decrement appeared when subjects 
had to detail which way the parameter was 
moving and its alert level, suggesting the MDO 
display required more processing (Stokes & 
Wickens, 1988, p. 414). No preference was 
indicated for the displays even though other 
research has shown that subjects prefer 
integrated displays (Jones, Wickens, & Deutsch, 
1990, p. 13). For relatively stable displays, i.e., 
displays that do not change often, some type of 
collocation may be beneficial because of same- 
object benefits (Davis, 2004). 

It is unclear whether collocation or pattern 
matching is driving the performance differences. 
Detecting a change from a typically stable 
condition suggests that collocation would be of 
benefit along with some type of pattern 
matching. This would indicate that the DC 
display should have shown superior performance 
because the information is more collocated with 
pattern matching available but with easy to 
discern parameter directional movement. 

In any case, the results indicate that while the 
MDO display is not as well liked or as good as 
the DC display, both collocated displays can be 
easily learned and used. The primary problem 
with the MDO display seems to be in 
determining whether a parameter is high or low, 
and the alert level. This aspect has to be learned 
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because, unlike the other two displays, the up 
and down direction does not indicate high and 
low and the alert range is not located at the top 
or bottom of the display. Therefore, a more 
realistic experiment using these three displays 
will be conducted in order to better determine 
the benefits of each type of display. 
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Tables 


Table 1. System Components with Their Parameters and Alert Levels 


Component and 
Parameter 

Alert Level 

Tank 


Temperature 

Advisory (high) 

Quantity 

Warning (low) 

Left Pump 


Pressure 

Warning (high) 

Temperature 

Caution (high) 

Right Pump 


Pressure 

Caution (high) 

Temperature 

Advisory (high) 

System 


Pressure 

Warning (high) 
Advisory (low) 

Temperature 

Caution (high) 


Table 2. Objective Experiment Questions and Answer Choices 


Question 

Answer Choices 

1 . This picture indicates a 

• Can change state 

component that 

(i.e., on & off) 

• Can NOT change 
state 

2. This picture indicates a 

• On 

component that is 

• Off 

• Failed 

3. This picture indicates 

• Pressure 

what type of parameter? 

• Temperature 

• Quantity 

4. This picture indicates a 

• High 

parameter that is 

• Low 


• Alert 

5. Which component is 

• System 

non-normal? 

• Left Pump 

• Right Pump 

• Tank 

• Don’t Know 

6. Which parameter is non- 

• Pressure 

normal? 

• Temperature 

• Quantity 

• Don’t Know 

7. The non-normal 

• High 

parameter is? 

• Low 

• Don’t Know 


• No Alert 

• Advisory 

• Caution 

• Warning 

• Don’t Know 
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Table 3. Subjective Experiment Questions and Answer Choices 


Question 

Answer Choice Range 
(9 equal graduations) 

1. To determine the component (System, Left Pump, Right Pump, Tank) 
affected is . . . 

Very Easy to Very Hard 

2. To determine the parameter (Pressure, Temperature, Quantity) affected is . . . 

Very Easy to Very Hard 

3. To determine the direction (Low, High) is . . . 

Very Easy to Very Hard 

4. To determine the alert level (No Alert, Advisory, Caution, Warning) is . . . 

Very Easy to Very Hard 

5. To determine the overall system state is . . . 

Very Easy to Very Hard 

6. Compared to the standard dial display, to determine the component (System, 
Left Pump, Right Pump, Tank) affected for the [collocated display] is ... 

Easier to Harder 

7. Compared to the standard dial display, to determine the parameter (Pressure, 
Temperature, Quantity) affected for the [collocated] display is ... 

Easier to Harder 

8. Compared to the standard dial display, to determine the direction (Low, 
High) affected for the [collocated] display is ... 

Easier to Harder 

9. Compared to the standard dial display, to determine the alert level (No Alert, 
Advisory, Caution, Warning) affected for the [collocated] display is ... 

Easier to Harder 

10. My overall preference for the Standard Dial display is . . . 

Much Less Preferred to 
Much More Preferred 

1 1 . My overall preference for the [collocated] display is . . . 

Much Less Preferred to 
Much More Preferred 


Table 4. Percent Correct on Identifying Non-Normal Parameter 


Display 

Percent Correct 

Baseline 

98 

DC 

94 

MDO 

85 

Display Order 

Beseline then DC 

93 

Baseline then MDO 

96 

DC then Baseling 

99 

MDO then Baseline 

88 


Table 5. Percent Correct on Identifying Non-Normal Parameter Dire ction 


Display 

Percent Correct 

Baseline 

98 

DC 

96 

MDO 

81 

Display Order 

Beseline then DC 

91 

Baseline then MDO 

88 

DC then Baseling 

100 

MDO then Baseline 

92 
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Table 6. Percent Correct on Identifying Non-Normal Parameter Alert 


Display 

Percent Correct 

Baseline 

98 

DC 

97 

MDO 

66 

Display Order 

Standard then DC 

93 

Standard then MDO 

89 

DC then Standard 

100 

MDO then Standard 

77 


Table 7. Percent Correct on Level of Correctness 


Display 

Percent Correct 

Totally 

Incorrect 

Component 

Correct 

Comp & Param 
Correct 

Comp, Param, 
Dir Correct 

Totally 

Correct 

Baseline 

2 

1 

1 

1 

94 

DC 

3 

2 

1 

0 

94 

MDO 

9 

10 

9 

18 

54 

Display Order 

Standard then DC 

5 

2 

4 

1 

88 

Standard then MDO 

6 

0 

8 

5 

81 

DC then Standard 

1 

1 

0 

0 

99 

MDO then Standard 

6 

10 

2 

13 

69 


Table 8. Percent Correct on Parameter Direction During Training 


Display 

Percent Correct 

Low 

High 

Alert 

Baseline 

100 

100 

97 

DC 

100 

100 

94 

MDO 

100 

88 

79 

Display Order 

Standard then DC 

100 

100 

100 

Standard then MDO 

100 

100 

100 

DC then Standard 

100 

100 

90 

MDO then Standard 

100 

90 

82 
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Figures 
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Figure 2. Baseline Display 
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Figure 3. Dial-on-Control Display 



Figure 4. Multi-Dimensional Object Display 
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Figure 5. MDO Display with Pressure Changes 



Figure 6. MDO Display with Temperature Changes 
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Figure 7. MDO Display with Quantity Changes 
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Figure 8. Example Timed Question 
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Correct (%) Percent Correct (%) Percent 


Baseline Display 


100 



Don't Know Pressure Quantity Temperature 


DC Display 



Don't Know Pressure Quantity Temperature 


MDO Display 

100 -i 1 1 



Don't Know Pressure Quantity Temperature 
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Figure 9. Correctness in Determining Component Parameter Affected 
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Display 



Parameter Direction = Low 
Parameter Direction = High 
Parameter = Alert 


Level of Correctness 

Figure 10. Overall Level of Correctness by Display Type 
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'igure 12. Percent Correct for Determing Parameter Direction by Display Type 
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Figure 13. Percent Correct for Determing Parameter Direction by Display Order 
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Appendix A 

Human Participants Informed Consent Statement 

Title of Research: 4DOF and Dial-on-Control Usability Study 

Principal Investigator: Anna C. Trujillo, Research & Technology Directorate, Crew Systems & 

Operations Branch, (757) 864-8047 


Informed Consent: 

Federal regulations require us to obtain signed consent for participation in research involving human participants. 
The purposes of this form are to give you information that may affect your decision whether to say YES or NO to 
participation in this research, and to record the consent of those who say YES. After reading the statement in IV 
below, if you wish to consent, please indicate so by signing this form. 

Statement of Procedure: 

Thank you for your interest in this research. By this time you should have been able to review the description and 
procedures of this study provided to you by mail or when you arrived. You will find a summary of the major 
aspects below, including the risks and benefits of participating. Please feel free to ask any questions about the 
procedures at any time. Carefully read the information provided below. IF YOU WISH TO PARTCIPATE in this 
study, then sign your name and date the form in the space provided. Also initial all pages at the bottom. Any 
information you provide will be kept in strict confidence to protect your privacy. 

I. I understand that: 

1 . This is a research study. 

2. I will participating in a study to evaluate new display concepts for future flight decks. 

3. The study will be performed on a computer workstation. The task will involve answering questions by 
using a mouse after looking at a picture for 3 seconds. 

4. I will receive training on the tasks. I will be allowed time to familiarize myself with the web interface. 

5. My participation in this research will involve 1 session of approximately 1 hour. 

II. Potential Risks 

1. I may experience fatigue using the mouse. 

2. No other risks have been identified for this study. 

3. In the unlikely event that you are injured or otherwise experience discomfort, the NASA Langley Research 
Center has a medical clinic available to you and emergency medical personnel and ambulance service is 
available to transport you to the nearest hospital. If you have questions about the research and your rights 
should you experience any injury, you may contact the principal investigators listed at the beginning of this 
document. 

III. Potential Benefits 

1 . I will derive no direct benefit from my participation in this study other than experience with a new avionics 
display that may be part of aviation platforms that I might fly in the future. 

2. The results of my participation may help lead to improved safety, efficiency, capacity, and expandability of 
the National Aerospace System. 

IV. Confidentiality 

1 . Any information you provide will be kept in strict confidence to protect your privacy. 

2. No personal data that you provide will be released in any form. 

3. Result data will be keyed only to a test subject number. Only the Principal Investigator will have 
knowledge of test subject identities. 

V. Compensation 

1 . There is no compensation for participating in this study. 

2. Civil servant volunteers participating in the research do so in their official capacity. 
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3. A civil servant injured as a result of participating in the research may apply for compensation through the 
Federal Workers Compensation System. 

4. Non-civil servant volunteers injured as a result of participating in this research may fde a claim under the 
Federal Tort Claims Act (FTCA) by filing a Standard Form 95. For additional information, you may 
contact the LaRC Office of Chief Counsel at 864-3221. 

4. For additional information, you may contact the LaRC Office of Fluman Resources at 864-3194. 

VI. Freedom to Withdraw 

All participation in this study is voluntary. You are free to withdraw from this study at any time without penalty or 
loss of benefits otherwise available to you. Furthermore, you are free not to answer any questions or respond to any 
experimental situations that you choose without penalty. 

It is OK for you to say NO. Even if you say YES now, you are free to say NO later, and walk away or withdraw 
from the study - at any time. Your decision will not affect your relationship with NASA Langley Research Center. 
Also, the Principal Investigator reserves the right to withdraw your participation in this study, at any time, if she 
observes potential problems with your continued participation. 

VII. Safety 

As a voluntary test subject participating in this research, 1 understand that: 

1 . NASA is committed to ensuring my safety, health, and welfare plus the safety and health of all others involved 
with the research. 

2. I should report any accident, injury, illness, and changes in my health condition, hazards, safety concerns, or 
health concerns to Anna C. Trujillo, (757) 864-8047. If I am unable to reach the above named individual(s) or 
am not satisfied with their response, I should contact the Langley Research Center Safety Office at (757) 864- 
7233 or the Chairperson of the Langley Research Center Institutional Review Board, Jeffrey S. Hill, (757) 864- 
5107. 

3 . If I detect any unsafe condition that presents an imminent danger to myself, or others, I have the right and 
authority to stop the activity or test. In such cases the Principal Investigator and associated research personnel 
will comply with my direction, stop the activity, and take action to address the imminent danger. 

VIII. Statement of Consent: 

I certify that I have read and fully understand the explanation of procedures, benefits, and risks 
associated with the research herein, and I agree to participate in the research described herein. 
My consent to participate is given voluntarily and without coercion or undue influence. I 
understand that I may discontinue participation at any time. I have been provided a copy of this 
consent statement. If I have any questions or modifications to this consent statement, they are 
written below: 


Participant’s Name Participant’s Signature 


Witness Signature 


Participant’s Address Date 


Participant’s Phone Participant’s Date of Birth 
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